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1  Summary 

Highly  complex  behavior  is  common  in  both  the  natural  and  technological  world.  Nonlinear  differential 
operators  play  an  essential  role  in  enabling  accurate  modeling  and  prediction  of  this  behavior.  Nonlin¬ 
ear  systems  theory  provides  a  mathematical  framework  for  the  analysis  and  design  of  networks  of  these 
operators,  thereby  providing  the  foundation  for  scientists  and  engineers  to  understand  and  control  this 
highly  complex  behavior.  This  project  is  primarily  concerned  with  the  development  of  analysis  and 
computational  tools  that  can  accurately  characterize  the  performance  of  specific  classes  of  nonlinear 
differential  operators  in  capturing  specific  behavioral  properties  of  interest.  A  secondary  concern  is  the 
development  of  controller  synthesis  tools  that  enable  the  design  of  networks  of  differential  operators  so 
as  to  yield  specific  behavioral  properties.  At  the  completion  of  this  project  after  three  years  of  fund¬ 
ing,  outcomes  of  this  project  include  the  development  of  new  theoretical  and  computational  tools  for 
performance  bound  verification,  tight  performance  bound  characterization,  and  controller  synthesis 
for  representative  behavioral  properties.  Integral-input-to-integral-output,  integral-input-to-output, 
and  input-to-state  stability  properties  have  been  specifically  considered.  Substantial  effort  has  been 
invested  in  the  development  of  tools  for  the  numerical  approximation  of  solutions  to  the  attendant 
optimization  and  optimal  control  problems.  This  includes  computational  tools  utilizing  approximating 
Markov  chain  methods  and  max-plus  methods.  The  research  undertaken  is  documented  in  26  scholarly 
publications  (17  published  or  accepted  to  be  published,  9  in  press  or  in  review),  and  communicated 
via  numerous  presentations  (including  14  invited)  at  internationally  renowned  meetings  and  academic 
institutions.  These  outcomes  are  summarized  in  Table  1.  Details  of  publications  /  presentations  cited 
therein  are  listed  in  Section  3. 


Tools 

Scholarly  publications  / 
invited  presentations 

Performance  bound  verification 

[A1-A3,  A6,  A9],  [B3,B4,B8,B9], 
[T3,  T5-T7,  T9,  Til,  T12,  T14] 

Tight  performance  bound 
characterization 

[Al, A5, A16],  [B1,B2,B5], 

[T5,  T7,  Til,  T12,  T14] 

Controller  synthesis 

[A4, A12, A13, A15],  [B8],  [T10] 

Computation 

[A2,  A3,  A7,  A8,  A10,  A13,  A15] , 
[B2,  B3,  B5-B7, B9] ,  [T1-T12,T14] 

Other 

[All, A14, A17],  [T13] 

Table  1:  Scholarly  publications  /  invited  presentations. 
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2  Description  of  outcomes 

The  primary  outcome  of  this  project  [R4]  is  the  development  of  analysis  and  computational  tools  for  the 
characterization  of  performance  of  nonlinear  differential  operators  with  respect  to  specific  behavioral 
properties.  A  secondary  outcome  is  the  development  of  control  design  tools  for  facilitating  the  design 
of  networks  of  nonlinear  differential  operators  to  achieve  specific  behaviors.  Specific  project  outcomes 
include  the  development  of  tools  for  (i)  performance  bound  verification,  (ii)  tight  performance  bound 
characterization,  (iii)  controller  synthesis  to  achieve  specific  performance  bounds,  and  (iv)  numerical 
computations  for  facilitating  (i)-(iii).  Specific  properties  classified  as  integral-input-to-output  stability, 
integral-input-to-state  stability,  and  input-to-state  stability  have  been  considered  in  detail.  Ongoing 
efforts  directed  towards  (iv)  continue  via  the  investigation  and  development  of  idempotent  methods 
for  the  efficient  solution  of  worst-case  analysis  and  optimal  control  problems.  These  efforts  form  the 
basis  of  a  recently  funded  three  year  AFOSR  /  AOARD  project  [R5] . 

2.1  Nonlinear  differential  operators 

Throughout  this  project,  a  specific  class  of  nonlinear  differential  operators  is  considered.  These  oper¬ 
ators  consist  of  systems  of  non-homogenous  nonlinear  ordinary  differential  equations  together  with  an 
output  map.  Elements  T,  in  this  class  of  operators  map  inputs  w  to  outputs  z,  as  illustrated  in  Figure 
1,  via  the  explicit  form 

_  (  x(t)  =  f{x{t))  +  g{x{t))  w{t) ,  x(0)=xo, 

1  z(t)  =  h(x(t )) , 


Input 

w 


Nonlinear 

differential 

operator 


Output 

z 

- ► 


Figure  1:  A  nonlinear  differential  operator  with  input  /  output  pair  ( w,z )  and  internal  state  x. 

in  which  x(-)  denotes  a  dynamically  evolving  internal  state  trajectory,  and  /,  g,  h  are  functions  with 
appropriate  regularity.  It  is  well  known  that  operators  of  this  form  can  exhibit  extremely  complex 
and  localized  dynamical  behavior,  characterized  by  the  existence  of  sources,  sinks,  saddles,  periodic 
orbits,  chaos,  and  so  on.  Numerous  stability  properties  have  been  formalized  in  the  literature  (for 
example,  [R1,R6,R9-R13],  [A3])  to  classify  this  behavior,  typically  via  finite  gain  properties  formulated 
in  terms  of  the  operator  inputs,  outputs,  and  internal  states. 

2.2  Behavior  and  performance  bounds 

Operators  of  the  form  of  (1)  may  be  classified  according  to  specific  behavioral  properties  by  certifying 
whether  corresponding  input-to-output  or  input-to-state  finite  gain  inequalities  hold.  Specific  finite 
gain  inequalities  of  interest  include  integral-input-to-integral-output  stability  (iliOS,  e.g.  [R9],  [B8], 
[A3])  and  integral-input-to-output  stability  (ilOS,  e.g.  [R9],  [A5],  [B2] ) ,  given  respectively  by 

INITIO, t]  ^  0(M)  +  7  (lMll?2[o,t])  ,  (2) 

\z(t)\  <  P(\x0\,t)  +7(IMUi,[o1i])  7  (3) 

where  /?,  7  are  comparison  functions  [R12],  Finite  gain  inequalities  of  this  form  can  be  used  to  prescribe 
input-output  or  input-state  constraints  on  the  dynamics  of  the  nonlinear  differential  operators  of 
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interest.  The  comparison  functions  /3,  7  involved  are  not  unique,  and  any  specific  comparison  function 
need  not  represent  a  tight  bound  in  the  respective  finite  gain  property.  Indeed,  a  set  IIs  of  comparison 
function  bounds  compatible  with  operator  £  and  a  specific  finite  gain  property  may  be  defined.  For 
example,  in  the  case  of  iliOSS  property  (2)  considered  in  [Al],  this  set  is  defined  explicitly  by 

nE  =  |(/3,  7)€£x£ 


iliOSS  property  (2)  holds  for  all  1 
w  €  «S?2[0,  t],  t  €  M>q,  x0  €  Mn  J  ’ 


where  1C  denotes  a  class  of  functions  0  :  M>o  M>o  that  are  right  continuous  at  0,  non-decreasing, 
and  satisfy  0(0)  =  0.  Using  this  notation,  it  is  evident  from  (2)  that  ((3  +  e,7  +  (5)  €  IIE  for  any 
(0,7)  €  ns,  e,  6  €  1C.  In  this  context,  performance  bound  verification  for  an  operator  £  concerns 
the  problem  of  determining  whether  a  specific  pair  (0, 7)  of  comparison  functions  resides  in  the  set 
IIE.  To  this  end,  it  is  convenient  to  expand  this  notation  further.  In  particular,  the  set  of  transient 
bounds  £>^(7)  C  1C  compatible  with  a  given  gain  bound  7  €  /C,  or  the  set  of  gain  bounds  C  1C 

compatible  with  a  given  transient  bound  (3  €  1C,  may  be  defined  in  terms  of  the  set  IIE  of  (4)  by 


£s(7)  =  |0  €  K 

<5S(0)  =  (7€/C 


{0  7)  G  ns 
{0  7)  €  ns 


(5) 

(6) 


Hence,  performance  bound  verification  may  be  posed  with  respect  to  a  fixed  (or  known)  gain  or 
transient  bound.  For  example,  with  7  €  1C  fixed,  inspection  of  (4)  and  (5)  indicates  that  (/ 3 , 7)  €  nE 
if  and  only  if  0  €  BE( 7).  Similarly,  tight  performance  bounds  for  operator  £  may  be  characterized 
in  terms  of  these  set- valued  objects  £>E  and  .  In  particular,  the  tightest  possible  transient  bound 
(32  :  M>o  i->-  M>0  for  a  given  gain  bound  7  €  1C,  and  the  tightest  possible  gain  bound  7^  :  M>o  >-)■  M>0 
for  a  given  transient  bound  (3  €  1C,  are  given  by 


P2(s)  =  inf  j/3(«) 

7*  (s)  =  inf  (7(5) 


/3eHs(7)|  , 

7€eS(/3)l  • 


(7) 

(8) 


These  tight  bounds  demonstrably  reside  in  1C,  see  [Al],  so  that  {02,  7),  {0  7* )  €  nE.  That  is,  these 
tight  bounds  satisfy  the  corresponding  finite  gain  property  (in  this  case,  iliOS  (2)).  Consequently, 
these  tight  performance  bounds  may  be  used  to  summarize  the  behavior  of  specific  operators  with 
respect  to  specific  finite  gain  properties.  For  example,  two  operators  satisfying  the  iliOS  property  (2) 
may  be  compared  from  the  point  of  view  of  input-to-output  gain  via  their  respective  tightest  possible 
gain  bounds  yf  for  the  same  transient  bound  {3  €  1C.  Such  tight  bounds  are  of  crucial  importance  in 
reducing  conservatism  in  small-gain  based  feedback  design  [B8]. 

2.3  Variational  performance  bound  verification 

Verification  describes  a  mechanism  for  testing  the  compatibility  of  a  specific  nonlinear  differential 
operator  with  a  specific  behavioral  property  [A3].  In  the  context  of  the  iliOS  property  (2)  described 
above,  verification  involves  the  confirmation  (or  otherwise)  that  a  specific  pair  of  comparison  functions 
(/5, 7)  resides  in  the  set  ns  of  all  such  pairs  for  a  specific  operator  £  (see  (4))  .  Mechanisms  for  verifying 
this  set  membership  for  iliOS  and  other  behavioral  properties  developed  in  this  project  appeal  to 
dissipative  systems  theory  [R14] .  In  particular,  connections  between  specific  behavioral  properties 
and  corresponding  dissipation  properties  are  established.  These  dissipation  properties  are  equivalent 
to  the  existence  of  a  solution  to  a  corresponding  Hamilton  Jacobi  Bellman  partial  differential  equation 
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(HJB  PDE)  or  dynamic  programming  principle  (DPP)  that  is  specific  to  each  property.  For  example, 
in  the  case  of  the  iliOS  property  (2)  above  (with  7  differentiable),  the  nonlinear  differential  operator 
E  satisfies  property  (2)  if  and  only  if  an  augmented  differential  operator  Ea  is  dissipative  [A3]  with 
supply  rate  r,  where 


'  x(t)  ' 

f(x(t))+ g(x(t))w(t) 

.  m 

[  M*)  I2  J 

z(t) 

h(x(t)) 

_  v(t)  . 

£(i) 

5 

O 

X 

? 

L  m  \ 

.  £  _ 

w. 


(9) 


=  7 '(rj)  |w;|2  —  \z\ 


It  has  been  shown  in  [A3,A6,A9]  that  this  dissipation  property  holds  if  and  only  if  a  viscosity  super- 
solution  V  of  the  HJB  PDE 


0  =  H{x,  a,  VxV(x,Z),  V^(s,0) 


(10) 


s 

(a)  Performance  bound  candidates  71  and  72. 


(b)  Solution  of  HJB  PDE  (10)  for  71. 


(c)  Solution  of  HJB  PDE  (10)  for  72. 


Figure  2:  Performance  bound  verification  via  solution  of  HJB  PDE  (10). 
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exists  [A3],  [R2],  in  which  H  denotes  the  Hamiltonian 


H{x,£,p,  q)  =  —\h(x)\2  -  (p,  f(x))  -  sup  {(p,  g(x)w)  +  (q  -  Y(0)  |^|2}  •  (H) 

weRs 

Consequently,  the  iliOS  property  (2)  may  be  verified  by  solving  a  specific  H JB  PDE  [A3] .  In  partic¬ 
ular,  (2)  holds  with  a  specific  comparison  function  bound  pair  (/3,'y)  if  the  HJB  PDE  (10)  (which  is 
parameterized  by  comparison  function  7)  admits  a  non-negative  solution  V  that  satisfies 

V(x,0)  <  0(\x\)  (12) 


for  all  x  €  Mn.  As  a  simple  illustration,  verification  of  the  iliOS  property  (2)  may  be  considered  for 
the  nonlinear  differential  operator  [A3] 


£  : 


/(*) 


-§  (®  +  1)  +  1 
53 

(  —  X5 


g(x)  =  1 , 


k  h(x)  =  x . 


x  <  — 1 , 

x  e  [-1, 1] , 

X  >  1  , 


(13) 


Two  candidate  performance  bounds  (71  valid  and  72  invalid)  for  the  comparison  function  7  in  (2)  are 
illustrated  in  Figure  2(a).  (The  tightest  possible  such  bound  7*  is  also  illustrated  for  comparison,  see 
Section  2.4.)  These  candidate  bounds  are  tested  via  computation  of  the  solution  V  of  the  HJB  PDE 
(10),  with  the  respective  solutions  illustrated  in  Figure  2(b)  and  (c).  Candidate  72  is  rendered  invalid 
by  strict  positivity  of  the  HJB  PDE  solution  at  the  origin  [A3].  However,  candidate  71  is  valid  for 
any  comparison  function  j3  €  /C  satisfying  (12),  for  example,  any  quadratic  function  (3{x)  =  a\x\2 
for  a  €  M>o  sufficiently  large.  Performance  bound  verification  for  integral  input-to-state  stability 
(iISS,  [R9])  and  related  behavioural  properties  may  be  considered  in  a  similar  way,  see  [A5,A7],  [B2,B9]. 

2.4  Tight  performance  bound  characterization 

Performance  characterization  with  respect  to  the  aforementioned  behavioural  properties  involves  the 
determination  of  tight  bounds  for  either  or  both  comparison  functions.  This  project  has  developed 
methods  for  the  variational  characterization  of  tight  performance  bounds  for  behavioral  properties 
including  iliOS,  iISS,  and  ISS,  see  [Al,  A2,  A5,  A7,  A10,  A16],  [B1  B3,B5,B9].  In  the  case  of  the  iliOS 
property  (2),  these  tight  bounds  are  defined  by  /3*  and  7*  of  (7)  and  (8)  respectively.  In  order  to 
facilitate  computation  of  these  tight  bounds,  variational  characterizations  for  /3*  and  7*  have  been 
developed  [A1,A5,A16],  [B1,B2,B5].  These  characterizations  are  given  by 

7 *(s)=  sup  \w{x,  Vs)  -/3(|x|)|  ,  (14) 

a:GRn  1  J 

P2{s)=  sup  sup  \w(x,  y/t) -7(0}  ,  (!5) 

|a:|<s£eR>o  J 


where  the  underlying  function  W  :  Rn+1  1— >  R  is  the  value  of  a  specific  optimization  problem  [A1,A16]. 
In  particular, 


W(x,  £)  =  sup  sup 

t> 0  ves?2[o,t] 


'\\%2i0  ,t] 


z  =  (£  o  S)  v 


(16) 


6 


£ 


4 


z 

W 

*  £ 

£ 

5  j 

Nonlinear  system  Energy  saturation 

under  test  IM|-Sf2[o,oo)  ^  lf°l 


Figure  3:  Operator  cascade  [A1,A6]  used  in  tight  performance  bound  characterization  for  iliOS  (2). 


where  £  denotes  the  nonlinear  differential  operator  of  (1)  under  test,  S  denotes  an  “energy  saturation” 
operator  [Al,  A16],  and  So  S  denotes  a  cascade  of  these  two  operators,  see  Figure  3.  The  specific 
operator  S  employed  in  [Al,  A16]  is  given  explicitly  by 

£(t)  =  -Z(t)\v{t)  |2,  £(0)  =  £, 

w(t)  =  V2  £(t)  v{t). 

(This  operator  implements  the  input  energy  constraint  ||tf  ||.sf2[oy]  <  £.)  Lower  bounds  for  yf  and  (32 
also  prove  useful.  These  lower  bounds,  denoted  by  7*  and  /?*,  are  defined  (respectively)  in  terms  of 
W  of (16)  by 

7*(s)  =  J'L(C) ,-v/s) , 

(3*(s)  =  sup  W(x,  0) 

|x|<3 

In  view  of  bounds  (14),  (15),  (18),  (19),  the  key  to  computation  of  tight  performance  bounds  is 
computation  of  the  value  function  W  of  (16).  Application  of  dynamic  programming  [R3]  to  (16) 
reveals  [Al]  that  IF  is  a  viscosity  solution  of  the  HJB  PDE 

0  =  H(x,  VxW(x,  0,  V^W(x,  0)  (20) 


(18) 

(19) 


uy> 


(a)  Value  function  W  of  (16). 


s 

Ql 

(b)  Tight  performance  bounds  7** ,  7*  of  (14),  (18). 


Figure  4:  Tight  performance  bound  characterization  via  solution  of  HJB  PDE  (20). 
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for  all  (x,£)  £  Mn  x  M^O)  subject  to  the  boundary  condition  W(-,0)  =  ||£ ^o||^f2r0  ^  parameterized 

by  the  initial  internal  state  of  operator  E.  Here,  wo  denotes  the  zero  input  to  operator  E,  while  H  is 
the  Hamiltonian 

H(x,£,p,q)  =  ~\h{x)\2  -  <p,  f{x))  -  sup  \(p,  y/2£g(x)v)  -  q£\v\2}  .  (21) 

Hence,  the  HJB  PDE  (20)  in  concert  with  the  variational  performance  bound  verification  results  of 
Section  2.3  can  be  used  to  compute  the  value  W  of  (16),  and  hence  the  tight  performance  bounds  of 
(14),  (15),  (18),  (19).  This  approach  may  be  demonstrated  [Al]  via  the  simple  nonlinear  differential 
operator  (13).  In  particular,  Figure  4(a)  illustrates  an  approximate  solution  of  the  HJB  PDE  (20) 
obtained  via  an  approximating  Markov  chain  method,  see  [Al],  [R7].  Figure  4(b)  illustrates  the 

oi 

obtained  tight  performance  bounds  7*  and  7*  of  (14)  and  (18).  Note  that  in  the  former  case,  the 
tight  gain  bound  obtained  is  computed  via  the  steps  7  — »  /3*  — >■  7^* .  Note  that  both  of  these  steps 
use  the  same  approximation  for  W  of  (16),  so  that  (20)  need  only  be  solved  once.  Here,  7  €  /C  is  an 
arbitrary  (but  valid)  comparison  function  bound  for  the  gain  7  in  (2). 

Further  connections  between  performance  bound  verification  and  the  characterization  of  tight  per¬ 
formance  bounds  can  also  be  established.  For  example,  it  may  be  shown  [A9]  that  the  respective 
solutions  V  and  W  of  the  HJB  PDEs  (10)  and  (20)  can  be  related  for  a  given  gain  bound  7  €  JC  via 

V(x,  £)  =  sup  {  W{x,  yfp)  -  7(£  +  p) }  +  7(C)  (22) 

for  all  (x,£)  €  Mn  x  M>o-  Furthermore,  the  verification  condition  (12)  indicates  that  the  tightest 
possible  transient  bound  01  given  7  €  K.  (as  defined  by  (7))  can  be  represented  equivalently  by 

/32(s)  =  sup  V{x,0)  (23) 

\x\<s 

for  all  s  €  M>o-  Indeed,  it  may  be  seen  that  combining  (22)  and  (23)  immediately  yields  the  variational 
characterization  (15). 

2.5  Controller  synthesis  to  achieve  specific  performance  bounds 

The  behavior  of  nonlinear  differential  operators  may  be  manipulated  via  the  introduction  of  feedback 
to  achieve  specific  performance  bounds.  By  exploiting  the  connection  between  dissipation  and  a 
specific  behavioral  property,  c.f.  performance  verification  (see  Section  2.3  above),  an  optimal  control 
problem  may  be  posed  whose  solution  implements  that  specific  behavioral  property  in  closed-loop.  The 
solution  of  this  optimal  control  problem  may  be  synthesized  via  solution  of  an  attendant  Hamilton- 
Jacobi-Bellman-Isaacs  (HJBI)  PDE.  Controller  synthesis  to  achieve  the  specific  iliOS  property  (2) 
in  this  way  is  investigated  in  [A4].  In  considering  feedback  interconnections  of  nonlinear  differential 
operators,  approaches  to  small-gain  based  control  design  to  achieve  specific  behavioral  properties 
such  as  iliOS  (2)  may  also  be  developed.  These  approaches  utilize  well-known  small-gain  conditions, 
formulated  in  terms  of  comparison  functions  derived  from  behavioral  properties  for  individual  open- 
loop  operators,  to  infer  performance  bounds  for  the  closed-loop  interconnections.  These  inferences 
concern  the  performance  of  feedback  interconnections  of  operators  [A4],  [B8],  and  that  of  networks  of 
operators  [A12,  A13,  A15].  In  applying  these  small-gain  conditions,  tighter  performance  bounds  for  the 
component  operators  leads  to  tighter  performance  guarantees  for  the  interconnections.  Consequently, 
small-gain  based  design  can  benefit  directly  from  the  characterization  of  tight  performance  bounds  of 
the  component  operators  summarized  in  Section  2.4. 


2.6  Numerical  methods  /  tools 

HJB  PDEs  of  the  form  (10)  and  (20)  rarely  admit  explicit  solutions.  Where  explicit  solutions  do  exist, 
they  typically  correspond  to  very  simple  operators.  For  example,  an  explicit  solution  W  :  M2  H >  K 
of  the  tight  performance  bounds  HJB  PDE  (20)  does  exist  in  the  case  of  scalar  linear  operators  £, 
see  [Bl].  By  virtue  of  (22),  this  means  that  an  explicit  solution  f:lx  M>q  h >  R  of  the  performance 
verification  HJB  PDE  (10)  often  also  exists  (depending  on  the  choice  of  7  €  1C).  However,  even  in 
this  seemingly  simple  case,  these  solutions  take  a  non-trivial  form  [Bl].  This  is  due  to  the  fact  that 
while  the  operator  £  is  linear,  the  required  cascade  operator  £  o  S  in  (16)  remains  nonlinear.  In 
general,  this  inherent  nonlinearity  acts  to  further  complicate  any  nonlinear  dynamics  in  £,  rendering 
explicit  solutions  to  these  HJB  PDEs  rare  indeed.  Consequently,  numerical  methods  are  crucial  to  the 
applicability  of  the  performance  verification  and  characterization  tools  developed.  The  approaches  to 
numerical  computation  considered  in  this  project  [R4]  may  be  classified  as  targeting  either  the  HJB 
PDE  in  question,  or  the  corresponding  dynamic  programming  principle.  In  particular,  approximating 
Markov  chain  methods  [R7]  and  preliminary  max-plus  (idempotent)  methods  [R8]  have  been  developed 
for  solving  specific  performance  verification  and  tight  performance  bound  characterization  problems. 
Respectively,  these  Markov  chain  and  max-plus  methods  are  documented  in  [Al,  A3-A6,  A16],  [B2] 
and  [A2,A7,A8,A10],  [B3,B6,B7,B9]. 
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